Unraveling motor imagery brain patterns using explainable artificial intelligence based on Shapley values.

BACKGROUND AND OBJECTIVE: Motor imagery (MI) based brain-computer interfaces (BCIs) are widely used in rehabilitation due to the close relationship that exists between MI and motor execution (ME). However, the underlying brain mechanisms of MI remain not well understood. Most MI-BCIs use the sensorimotor rhythms elicited in the primary motor cortex (M1) and somatosensory cortex (S1), which consist of an event-related desynchronization followed by an event-related synchronization. Consequently, this has resulted in systems that only record signals around M1 and S1. However, MI could involve a more complex network including sensory, association, and motor areas. In this study, we hypothesize that the superior accuracies achieved by new deep learning (DL) models applied to MI decoding rely on focusing on a broader MI activation of the brain. Parallel to the success of DL, the field of explainable artificial intelligence (XAI) has seen continuous development to provide explanations for DL networks success. The goal of this study is to use XAI in combination with DL to extract information about MI brain activation patterns from non-invasive electroencephalography (EEG) signals. METHODS: We applied an adaptation of Shapley additive explanations (SHAP) to EEGSym, a state-of-the-art DL network with exceptional transfer learning capabilities for inter-subject MI classification. We obtained the SHAP values from two public databases comprising 171 users generating left and right hand MI instances with and without real-time feedback. RESULTS: We found that EEGSym based most of its prediction on the signal of the frontal electrodes, i.e. F7 and F8, and on the first 1500 ms of the analyzed imagination period. We also found that MI involves a broad network not only based on M1 and S1, but also on the prefrontal cortex (PFC) and the posterior parietal cortex (PPC). We further applied this knowledge to select a 8-electrode configuration that reached inter-subject accuracies of 86.5% ± 10.6% on the Physionet dataset and 88.7% ± 7.0% on the Carnegie Mellon University's dataset. CONCLUSION: Our results demonstrate the potential of combining DL and SHAP-based XAI to unravel the brain network involved in producing MI. Furthermore, SHAP values can optimize the requirements for out-of-laboratory BCI applications involving real users.

Assessing the Potential of Brain-Computer Interface Multiplayer Video Games using c-VEPs: A Pilot Study.

Video games have become a common and widespread form of entertainment, while non-invasive brain-computer interfaces (BCI) are emerging as potential alternative communication technologies. Combining BCIs and video games can enhance the gaming experience and make it accessible to motor-disabled individuals. Recently, code-modulated visual evoked potentials (c-VEP) have been proposed as a novel control signal able to achieve high performance with short calibration times. However, there are still no video games that use c-VEPs as a control signal. The aim of this pilot study is to develop an implementation of the 'Connect 4' multiplayer video game using a c-VEP-based BCI and test it with 10 healthy users. Participants were paired to compete in matches and carried out individual tasks. The results showed that the participants were able to control the game with an average accuracy of 94.10% and a selection time of 5.25 seconds per command, outperforming previous approaches. This suggests that the proposed video game is feasible and c-VEPs can provide smooth BCI control.

Nonparametric Early Stopping Detection for c-VEP-based Brain-Computer Interfaces: A Pilot Study.

Brain-computer interface (BCI) systems based on code-modulated visual evoked potentials (c-VEP) stand out for achieving excellent command selection accuracies with very short calibration times. One of the natural steps to democratize their use in plug-and-play environments is to develop early stopping algorithms. These methods allow real-time detection of the minimum number of code repetitions needed to provide reliable selections. However, such techniques are scarce in the current state-of-the-art for c-VEP-based BCI systems based on the classical circular shifting paradigm. Here, a novel nonparametric early stopping method is proposed, which approximates the distribution of unattended commands to a normal distribution and issues a selection when the correlation of the command is considered an outlier. The proposal has been evaluated offline with 15 healthy users, achieving an average accuracy of 97.08% and a speed of 1.37 s/command. Likewise, the algorithm has also been evaluated with an additional user in an online way, as a proof of concept to validate its technical feasibility, achieving an average accuracy of 96.88% with a speed of 1.67 s/command. These results suggest that the real time application of the proposed algorithm is feasible, significantly reducing the required selection time without compromising accuracy.

Combining brain-computer interfaces and multiplayer video games: an application based on c-VEPs.

Introduction and objective: Video games are crucial to the entertainment industry, nonetheless they can be challenging to access for those with severe motor disabilities. Brain-computer interfaces (BCI) systems have the potential to help these individuals by allowing them to control video games using their brain signals. Furthermore, multiplayer BCI-based video games may provide valuable insights into how competitiveness or motivation affects the control of these interfaces. Despite the recent advancement in the development of code-modulated visual evoked potentials (c-VEPs) as control signals for high-performance BCIs, to the best of our knowledge, no studies have been conducted to develop a BCI-driven video game utilizing c-VEPs. However, c-VEPs could enhance user experience as an alternative method. Thus, the main goal of this work was to design, develop, and evaluate a version of the well-known 'Connect 4' video game using a c-VEP-based BCI, allowing 2 users to compete by aligning 4 same-colored coins vertically, horizontally or diagonally. Methods: The proposed application consists of a multiplayer video game controlled by a real-time BCI system processing 2 electroencephalograms (EEGs) sequentially. To detect user intention, columns in which the coin can be placed was encoded with shifted versions of a pseudorandom binary code, following a traditional circular shifting c-VEP paradigm. To analyze the usability of our application, the experimental protocol comprised an evaluation session by 22 healthy users. Firstly, each user had to perform individual tasks. Afterward, users were matched and the application was used in competitive mode. This was done to assess the accuracy and speed of selection. On the other hand, qualitative data on satisfaction and usability were collected through questionnaires. Results: The average accuracy achieved was 93.74% ± 1.71%, using 5.25 seconds per selection. The questionnaires showed that users felt a minimal workload. Likewise, high satisfaction values were obtained, highlighting that the application was intuitive and responds quickly and smoothly. Conclusions: This c-VEP based multiplayer video game has reached suitable performance on 22 users, supported by high motivation and minimal workload. Consequently, compared to other versions of "Connect 4" that utilized different control signals, this version has exhibited superior performance.

ITACA: An open-source framework for Neurofeedback based on Brain-Computer Interfaces.

BACKGROUND AND OBJECTIVE: Neurofeedback (NF) is a paradigm that allows users to self-modulate patterns of brain activity. It is implemented with a closed-loop brain-computer interface (BCI) system that analyzes the user's brain activity in real-time and provides continuous feedback. This paradigm is of great interest due to its potential as a non-pharmacological and non-invasive alternative to treat non-degenerative brain disorders. Nevertheless, currently available NF frameworks have several limitations, such as the lack of a wide variety of real-time analysis metrics or overly simple training scenarios that may negatively affect user performance. To overcome these limitations, this work proposes ITACA: a novel open-source framework for the design, implementation and evaluation of NF training paradigms. METHODS: ITACA is designed to be easy-to-use, flexible and attractive. Specifically, ITACA includes three different gamified training scenarios with a choice of five brain activity metrics as real-time feedback. Among them, novel metrics based on functional connectivity and network theory stand out. It is complemented with five different computerized versions of widespread cognitive assessment tests. To validate the proposed framework, a computational efficiency analysis and an NF training protocol focused on frontal-medial theta modulation were conducted. RESULTS: Efficiency analysis proved that all implemented metrics allow an optimal feedback update rate for conducting NF sessions. Furthermore, conducted NF protocol yielded results that support the use of ITACA in NF research studies. CONCLUSIONS: ITACA implements a wide variety of features for designing, conducting and evaluating NF studies with the goal of helping researchers expand the current state-of-the-art in NF training.

MEDUSA©: A novel Python-based software ecosystem to accelerate brain-computer interface and cognitive neuroscience research.

BACKGROUND AND OBJECTIVE: Neurotechnologies have great potential to transform our society in ways that are yet to be uncovered. The rate of development in this field has increased significantly in recent years, but there are still barriers that need to be overcome before bringing neurotechnologies to the general public. One of these barriers is the difficulty of performing experiments that require complex software, such as brain-computer interfaces (BCI) or cognitive neuroscience experiments. Current platforms have limitations in terms of functionality and flexibility to meet the needs of researchers, who often need to implement new experimentation settings. This work was aimed to propose a novel software ecosystem, called MEDUSA©, to overcome these limitations. METHODS: We followed strict development practices to optimize MEDUSA© for research in BCI and cognitive neuroscience, making special emphasis in the modularity, flexibility and scalability of our solution. Moreover, it was implemented in Python, an open-source programming language that reduces the development cost by taking advantage from its high-level syntax and large number of community packages. RESULTS: MEDUSA© provides a complete suite of signal processing functions, including several deep learning architectures or connectivity analysis, and ready-to-use BCI and neuroscience experiments, making it one of the most complete solutions nowadays. We also put special effort in providing tools to facilitate the development of custom experiments, which can be easily shared with the community through an app market available in our website to promote reproducibility. CONCLUSIONS: MEDUSA© is a novel software ecosystem for modern BCI and neurotechnology experimentation that provides state-of-the-art tools and encourages the participation of the community to make a difference for the progress of these fields. Visit the official website at https://www.medusabci.com/ to know more about this project.